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ABSTRACT 


The  likelihood  ratio  principle  is  employed  to  suggest  a  non- 
parametric  test  for  testing  equality  of  two  distributions  against 
a  stochastic  ordering  alternative.  The  test  appears  to  be  robust 
against  a  wide  range  of  alternatives.  Percentage  points  for 
sample  sizes  less  than  or  equal  to  twenty  are  provided  as  well  as 
a  comparison  of  power  values  for  the  Kolmogorov-Smirnov  and  Mann- 
Whitney-Wilcoxon  tests. 


A  Robust  Nonparametric  Likelihood  Ratio  Test 


1.  Introduction. 

The  likelihood  ratio  principle  for  obtaining  test  statistics 
has  proven  to  be  a  popular  and  fruitful  method  in  parametric 
situations.  It  leads  to  such  appealing  tests  as  the  Student  t 
test  when  testing  for  equality  of  means  (variances  assumed  equal) 
and  the  F  test  when  tes.ting  for  equality  of  variances  when  under¬ 
lying  normal  distributions  are  assumed. 

In  this  paper,  it  is  proposed  that  the  likelihood  ratio 
principle  be  used  in  a  nonparametric  setting  to  construct  two 
sample  test  statistics  which  will  compare  favorably  with  such 
nonparametric  tests  as  the  Mann-Whitney-Wilcoxon  and  Kolmogorov- 
Smirnov  tests. 

Exact  small  sample  tables  are  provided  in  Section  3  as  well 
as  a  comparison  of  power  with  standard  nonparametric  tests. 
Theorems  on  symmetry  and  unbiasedness  are  proven  in  the  appendix. 

2.  The  Tests. 

Let  us  assume  that  we  have  independent  random  samples  from 
two  discrete  populations  with  respective  survival  functions 
(1  -  cdf's)  P(t)  and  Q(t) .  (For  convenience  we  assume  that 
P(0)  =  Q(0)  =1.)  If  we  wish  to  test  the  hypothesis  of  stochastic 
ordering  versus  all  other  alternatives,  then  our  hypotheses  may  be 

st 

H Q:  P  >  Q  (P(t)  >  Q ( t )  v  t) 
st 

P  >  Q  (P(t)  <  Q ( t )  for  at  least  one  t) . 


written  as 


2 


To  construct  a  likelihood  ratio  test  we  will  need  to  find 
title's  (maximum  likelihood  estimates)  of  P  and  Q  subject  to  a 
stochastic  ordering  restriction.  Thus  if  our  first  sample  contains 
6^  observations  at  x^,  <$2  observations  at  X2*  etc.,  and  the  second 
sample  d^  observations  at  y^,  d^  observations  at  y etc.,  we  must 
be  able  to  maximize  the  expressions 

nl  5.  ml  d. 

L(P,Q)  =  n  (P(x.-)  -  P(x.))  1  n  (Q (y .  -)  -  Q (y . ) )  3  (2.1) 

i=l  1  1  j=l  3  3 

st 

subject  to  the  restriction  P  s  Q. 

This  maximization  is  discussed  in  Brunk,  et.  al.  (1966)  and  in 

more  tractable  form  in  Dykstra  (1980).  To  obtain  the  desired  maxi¬ 

mal  value  of  (2.1),  proceed  as  follows: 

a)  Let  s.  <  s0  <  ...  <  s  denote  the  distinct  combined  values 

■L  z  m 

of  the  xi’s  and  yi ' s  (combined  observed  values). 

b)  Now  redefine  <5^(d^)  to  be  the  number  of  observations  from 
the  first  (second)  sample  at  the  point  s^ 

m  m 

c)  Let  n.  =  Z5.  (m.=  Id.)  denote  the  number  of  observa- 

3  i=j  1  3  i=j  1 

tions  from  the  first  (second)  sample  at  least  as  large  as  s ^ . 

d)  For  1  s  a  <  b  <  m,  define  k  ,  +  by 

+  b  b  b 

k  .  =  max{  (m  £  <5  .  -  n  £d.)(£(d.  +5.))  ,0). 

a,b  a  a  1  a  a  1  a  1  1 

(Note  that  k  , +  is  a  weighted  average  of  m  and  -n  if  positive.) 


3 


e)  For  1  s  i  <  m,  define 


k. 

1 


min  max  k 
a<i  isb  a,b 
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Pi  = 


n  .  -  6  .  +  k  . 

— - - ,  and 

n  .  +  k  . 

i  i 


A 


A  A  A 

If  =  m.  >  0,  =  0/0  is  indeterminate.  In  these  cases  we  must 

A  As  /V 

define  «  p. .  If  m.  =  0,  we  define  q.  to  also  be  zero. 

1  st 

f)  The  maximal  value  of  (2.1)  subject  to  P  >  Q  is  then  given 
by 


a  a  m  a  5  .  a  n  . -6  .  a  d  >  a  m , * d . 

L(P,Q)  =  n  (1  -  p.)  1  p.  1  X(1  -  q.)  1  q.  1  1  (2.2) 

i=l  1  1  11 

where  we  adopt  the  convention  that  0^=1. 

It  is  well  known  that  the  unrestricted  maximum  of  (2.1)  occurs 
by  putting  equal  probability  at  the  observations  from  each  sample. 
This  is  equivalent  to  requiring  the  k^'s  in  e)  to  be  zero.  Thus  if 

*  ni  "  *  mi  ’  di 

p^  =  — — -  and  q^  —  ,  i  —  1#  •  . .  ^m. 


we  have 


sup  L(P,Q)  =  L(P *,Q*) 
P»Q 


m 


6  . 


n  (i  -  p4> 

i=l  1 


•  . n . -6  . 
1*1  l 


(i  - 


*di 


*m.-d. 
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The  likelihood  ratio  principle  asserts  that  HQ  should  be  rejected 
if  » 

sup  L(P,Q) 
st 

x  =  - 

sue  L(P,Q) 

P,Q 

(2.3) 


is  sufficiently  small.  Equivalently,  since  X  may  become  very  small 
with  increasing  sample  size,  we  use  as  our  test  statistic 

T  =  -  2 ZnX 

and  reject  HQ  if  T  becomes  too  large.  Since  the  size  of  such  a 
test  with  critical  point  tQ  is  given  by 

sup  Pr(T  >  tQ) , 
st 

P  >  Q 

it  is  fortunate  that  if  this  supremum  is  attained,  it  is  attained 
in  the  class  of  distributions  where  P  =  Q  (assuming  continuous 
distributions) . 

Theorem  2.1.  Assuming  P  and  Q  are  continuous, 

sup  Pr (T  >  t0)  =  sup  Pr (T  >  t  ) 
st  P  =  q  0 

P  S  Q 

for  all  tg. 

Proof.  (See  Appendix.) 


a  MP.Q) 

*  * 

L(P  »Q  ) 
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If  we  sample  from  continuous  distributions,  then  with  proba¬ 
bility  one,  all  the  6  Vs  and  dVs  will  be  zero  or  one.  Since  the 

n.'s,  m.'s,  6.'s  and  d.'s  can  all  be  determined  from  the  ranks  of 
X  I  1  1 

the  combined  samples,  the  test  statistic  T  is  a  function  of  the 
combined  ranks,  and  hence  the  proposed  test  is  nonparametric  in 
nature.  While  we  realize  that  the  test  was  derived  under  the 
assumption  of  discrete  distributions,  since  a  continuous  distribu¬ 
tion  may  be  approximated  arbitrarily  closely  by  a  discrete  distri¬ 
bution,  we  feel  the  test  is  still  reasonable  in  the  continuous 
setting. 

Note  that  if  we  consider  only  continuous  P  and  Q,  the  supremum 

in  Theorem  2.1  is  attained  for  any  P  =  Q.  Because  of  this,  it 

seems  inherently  reasonable  to  restrict  the  class  of  alternatives 

st  st 

to  a  particular  subset  of  P  2  Q,  namely  P  <  Q,  and  use  the  same 
test  statistic  for  testing 

H'q:  P  =  Q 
st 

vs  H':  p  <  Q  (P(t)  <  Q{t)  for  all  t  and  P(t)  <  Q(t)  for  some  t) . 

Similar  arguments  to  those  used  in  Theorem  2.1  will  suffice 
to  show  that  the  proposed  test  is  unbiased. 

Corollary  2.1.  For  all  tQ,  and  continuous  P  and  Q, 

Inf  Pr (T  £  tQ)  =  inf  Pr(T  >  tQ)  . 

st  P  =  Q 

P  <  Q 


There  exists  some  symmetry  properties  between  =  IS.  and 
m 

m.  =  Id.  which  are  not  readily  apparent.  In  particular,  we  have 
A  i=l  1 

the  following  theorem. 

Theorem  2.2.  There  exists  a  1  -  1  mapping  between  those  strings 
(of  x's  and  y's)  containing  n^x's  and  m^y's  and  those  strings 
containing  m^x's  and  n^y's  such  that  corresponding  strings  have 
the  same  value  of  X  (and  hence  T) . 

Proof .  (See  Appendix.) 

Since  the  number  of  permutations  n^x's  and  m^y's  is  the  same 
as  for  m^x's  and  n^'s,  and  since  if  P  =  Q  (continuous),  all 
permutations  are  equally  likely,  we  have  the  following  corollary. 
Corollary  2.1.  If  P  =  Q  (continuous) ,  the  distribution  of  T  is 
symmetric  in  n^  and  n^. 

This  result  halves  the  work  of  finding  critical  points  for 
the  null  distribution  of  T. 

3.  Distribution  of  the  Test. 

Robertson  and  Wright  (1980)  have  derived  the  asymptotic  distri¬ 
bution  of  T  under  H  when  P  =  Q  are  of  the  discrete 

o 

type  with  only  a  finite  number  of  points  of  positive  probability. 

They  have  shown  that  asymptotically  the  survival  function  of  T 

(under  H  >  is  a  weighted  average  of  chi-square  survival  functions, 
o 

Wolfe  and  Lee  (1976)  have  discussed  a  Mann-Whitney-Wilcoxon 

A  A 

type  test  based  upon  the  restricted  mle's  P  and  Q  obtained  under 
the  stochastic  ordering  restriction,  but  were  unable  to  obtain  the 
asymptotic  distribution. 

To  obtain  the  null  distribution  of  T  for  the  case  P  =  Q 
(continuous)  for  small  sample  sizes,  we  have  used  the  computer  to 
enumerate  all  possible  rankings  of  the  samples  along  with  the 


corresponding  values  of  the  statistic  T.  From  these  we  have 
computed  the  .90,  .95  and  .99  percentage  points  of  the  null 
distribution  of  T.  For  values  of  n  and  m  too  large  to  consider 
all  rankings,  we  have  simulated  the  percentage  points  based  upon 
4000  independent  observations  for  each  value  of  n  and  m  through 
20.  The  percentage  points  are  displayed  in  Table  1. 

We  have  been  unable  to  determine  the  asymptotic  distribution 

of  T. 

4.  Power  of  the  Test. 

To  obtain  some  feeling  for  the  power  of  our  proposed  test 
compared  to  the  power  of  the  Mann-Whitney  and  the  Kolmogorov- 
Smirnov  tests,  we  simulated  observations  from  various  distributions. 
We  considered  sample  sizes  of  (5,5)  ,  (5,10)  and  (5,15)  and  generated 
1000  observations  to  obtain  each  empirical  estimate  of  the  power 
curve.  For  each  test  procedure,  we  chose  our  critical  points  to 
make  the  size  of  the  test  as  close  to  .05  as  possible  without  ex¬ 
ceeding  it.  Thus  if  we  were  unable  to  construct  a  test  whose  size 
was  near  .05,  we  would  expect  the  power  function  of  that  test  to 
be  lower.  This  happened  sometimes  with  the  Kolmogorov-Smirnov  test. 

We  constructed  our  alternatives  to  be  basically  of  three  types. 
The  first  type  consisted  of  standard  families  of  distribution  which 
naturally  gave  rise  to  a  stochastic  ordering.  The  families  of  dis¬ 
tributions  which  we  considered  were  the  normal,  the  exponential  and 
the  uniform  distributions. 

For  testing  normality  against  a  one-sided  shift  alternative, 
we  know  the  Mann-Whitney  test  is  very  good.  However,  the  new  test 
performs  nearly  as  well.  Values  of  the  power  function  for  the 
normal  distribution,  as  well  as  stochastically  ordered  alternatives 
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for  the  exponential  an  1  uniform  di stributions,  are  given  in  Table  2. 

In  each  of  these  situations,  the  new  test  is  nearly  as  powerful  as 

the  Mann-Whitney  test,  and  usually  more  powerful  than  the  t 

Kolmogorov-Smirnov  test. 

Of  course  stochastically  ordered  alternatives  may  occur  in 
many  ways.  One  way  is  to  have  Lehmann  type  alternatives,  i.e.,  the 
alternative  cdf  is  a  fixed  power  of  the  null  hypothesis  cdf.  We 
considered  this  kind  of  alternative  for  cases  where  the  null  hypothe¬ 
sis  was  an  exponential  (y  =  1)  distribution  and  the  case  where  it 
was  a  uniform  [0,1]  distribution.  A  Monte  Carlo  power  study  ob¬ 
tained  from  1000  independent  samples  is  presented  in  Table  3.  Under 
these  alternatives  the  new  test  performs  better  than  the  Mann- 
Whitney  for  the  sample  sizes  of  (5,5),  and  nearly  as  well  for  the 
sample  sizes  of  (5,10)  and  (5,15).  It  performs  uniformly  better 
than  the  Kolmogorov-Smirnov  test  for  these  alternatives. 

Finally  we  should  note  that  since  T  was  derived  as  a  test 

st 

against  the  alternatives  P  £  Q,  it  will  still  have  some  power  in 
detecting  differences  between  distributions  whose  cdf's  cross. 

The  Mann-Whitney  and  Kolmogorov-Smirnov  tests  usually  have  little 
power  in  these  situations.  Table  4  gives  power  values  for  uniform 
and  normal  distributions  with  mean  0  when  the  variance  is  allowed  to 
vary.  Finally,  Table  5  shows  power  values  when  the  coefficient  of 
variation  (y/o)  if  fixed  at  .5,  and  then  y  is  allowed  to  change.  We 
see  in  Tables  4  and  5  that  T  outperforms  the  other  tests  in  this 
situation . 

An  extensive  power  study  of  numerous  situations  indicates  that 
the  test  statistic  T  performs  well  under  a  wide  range  of  circum¬ 
stances.  For  small  nearly  equal  sample  sizes,  it  does  especially  well. 
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5 .  Appendix. 

*  *  1  n  1  m 

Proof  (of  Theorem  2.1).  Since  L(P  ,  Q  )  =  (— )  (i)  is  free  of 
-  n  m 

(X^#...»Xn  Y^ , . . . ,  Y  )  a.s.,  T  is  a  nonincreasing  function  of 

sup  L(P,Q)  a.s.  However,  as  discussed  in  Dykstra  (1980), 
st 

P  >.  Q 


sup  L(P,Q)  =  sup  L  (P)L  (Q)  (A.l) 

st  st  #st  1  A 

P  >  Q  P  >  R  ;>  Q 


* 

where  R  is  the  empirical  survival  function  of  the  combined  sample 

and  L.d^)  -*-s  the  likelihood  of  the  first  (second)  sample. 

Now,  for  fixed  yn ,  y„,...,y 
Li  m 


sup  L_(Q) 
*  st 
R  2  Q 


(A. 2) 


is  nondecreasing  in  x^  since  R  is  nondecreasing  in  x^.  Consider 

now  the  jump  in  P  at  the  point  x,  =  s .  say.  From  earlier  considera- 

k  3 

tions  this  is 


n .  -  1  +  k . 
n  — - - — i 

i<  j  n  .  +  k  , 

J  l  i 


n  .  +  k  . 
3  3 


Now  P  z  R  implies  that 


n 

i<  j 


1  +  k. 

l 


+  k . 

l 


> 


n .  + 
_J _ 


m . 
-J. 


n 


1 


+ 


m 


1 


(A. 3) 


Moreover 


min 


max 


a  <  j  j  <  b 


'a,b 


max  k .  , 
j  <  b  ]'b 


m , 


so  that 


(A. 4) 


- > - .  (A. 5) 

n  .  +  k  .  n  .  +  m . 

3  3  3  3 


Combining  (A. 4)  and  (A. 5),  we  see  that  the  probability  which  P 

* 

places  at  x^  is  never  less  than  the  probability  R  places  at  x^. 
Suppose  now  that  P  denotes  the  survival  function  which  is 


identical  to  P  except  it  places  the  probability  which  P  puts  at  x^ 
on  the  value  x^  +  A  (A  >  0#  x^  +  A  ^  xj  ^or  j)  • 

Thus  if  R  denotes  the  combined  empirical  survival  function 
with  x^  replaced  by  +  A,  we  must  have 

P(t)  >  R(t)  for  all  t 


(since  P  is  increased  by  at  least  as  much  as  R  over  the  interval 
[Xj,,  x^  +  A)  and  P  and  R  are  unchanged  elsewhere)  . 

However,  the  likelihood  under  P  and  P  are  equivalent.  Thus 


sup  *L  (P)  <  sup^L..  (P)  (A. 6) 

P  >  R  P  >  R 


where  the  first  likelihood  is  evaluated  at  x. , . . . ,x.  , . . . ,x  and  the 

±K  n 

sscond  fit  y  ^  m  • »  /  *  *  *  *  9  * 

Thus,  combining  (A. 2)  and  (A. 6),  we  see,  in  light  of  (A.l)  that 
y^,...,ym)  is  a  nonincreasing  function  of  each  x^  (all 
other  variables  held  constant) ,  and  hence 


I  (x 

IT 

is  nonincreasing  in  each  x^. 
integral 


I  •••  /  4 


Now  write  p (t 


•V 

s  ‘o’ 


as  an  iterated 
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P(T  >  tQ) 


E  **•  E  E  I(X1 . V  P 

X1  V  (T  *  t0) 


(A. 7) 


and  note  that  E  I  (X^,...,X  ,  Y) 
Y  (T  2  tQ) 


is  nonincreasing  in  each  , 


Since  it  is  well  known  that  for  nonincreasing  g  and  stochasti- 

st 

cally  ordered  survival  functions  P  >  Q 


00  00 

|g(t) d  P  (t)  <  jg (t)  d  Q ( t ) 
—  00  —00 


it  clearly  follows  that  sup  P(T  ^  t^) 

st  0 

P  >  Q 

must  occur  when  P  =  Q. 

Proof  (of  Theorem  2.2).  Since  the  value  of  A  (and  also  T)  depends 

only  upon  the  ranks,  we  may  assume  that  the  combined  sample  con- 

1  2 

sists  of  the  points  - ,  - ,  ...,1.  Then,  letting 

nl  +  mi  nl  +  ml 
n  =  n1  +  n^,  we  can  write 


A  =  C  sup 

P(x)  >  1  -  X  S  Q 


jjKHr1)  -  4)]  ‘['(^-r) '  =4)]  1 


(A. 


6  . 

l 


=  C  sup 
Q*>  1  -  x>  P1 


A[p'<44)  - p*  MB-r) 


d. 

i 


for  a  suitable  constant  C  where 


P*(x)  =  1  -  P(1  -  x) ,  and 
Q* (x)  =  1  -  Q(1  -  x) . 


k 


12 


*  * 

Noting  that  P  {Q  )  is  a  survival  function  iff  P(Q)  is,  it  is 

apparent  that  (A. 9)  is  of  the  exact  same  form  as  (A. 8) . 

Thus  if  two  strings  are  such  that  =  d^f  ^  and 

df^  =  <5^.,  they  must  give  the  same  value  of  X.  This  of  course 
i  n**i 

happens  if  a  particular  string  of  x's  and  y's  is  written  in  reverse 
order,  and  then  the  x's  replaced  by  y's  and  vice  versa.  This  then 
defines  the  1-1  mapping  mentioned  in  Theorem  2.2. 
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Table  3 


Estimated  Power  of  Two-Sample  One-Sided  Tests 


P:  U[0, 

1] 

Q:  UfO, 

l)k 

n 

m 

k 

K  -  S 

M  -  W 

T 

5 

5 

1.00 

.0340 

.0420 

.0450 

1.50 

.1060 

.1360 

.1480 

2.50 

.2460 

.3020 

.3240 

3.50 

.3830 

.4360 

.4730 

5.00 

.5410 

.5920 

.6310 

5 

10 

1.00 

.0330 

.0570 

.0630 

1.50 

.0920 

.1590 

.1470 

2.50 

.2640 

.3770 

.3620 

3.50 

.4130 

.5410 

.5130 

5.00 

.6080 

.7370 

.7090 

5 

15 

1.00 

.0160 

.0410 

.0450 

1.50 

.0760 

.1320 

.1360 

2.50 

.2370 

.3910 

.3700 

3.50 

.4470 

.6340 

.5830 

5.00 

.6360 

.8030 

.7800 

P:  exp(l) 

Q:  [exp(l)]k 

n 

m 

k 

K  -  S 

M  -  W 

T 

5 

5 

1.00 

.0410 

.0420 

.0510 

1.50 

.1040 

.1200 

.1380 

2.50 

.2130 

.2790 

.2940 

3.50 

.3890 

.4420 

.4740 

5.00 

.5380 

.5930 

.6350 

5 

10 

1.00 

.0320 

.0600 

.0550 

1.50 

.0780 

.1370 

.1350 

2.50 

.2550 

.3760 

.3370 

3.50 

.4180 

.5530 

.5270 

5.00 

.6090 

.7450 

.7050 

5 

15 

1.00 

.0240 

.0490 

.0480 

1.50 

.0560 

.1360 

.1320 

2.50 

.2480 

.4060 

.3720 

3.50 

.4340 

.6140 

.5870 

5.00 

.6750 

.8260 

.7990 

Table  4 


Estimated  Power  of  Two-Sample  One-Sided  Tests 
P:  U [-1 ,  1] 

Q:  ui-e,  e] 


n 

m 

0 

K  -  S 

M  -  W 

T 

5 

5 

1.00 

.0410 

.0480 

.0490 

.80 

.0360 

.0440 

.0430 

.60 

.0580 

.0530 

.0640 

.40 

.0770 

.0430 

.0820 

.20 

.1310 

.0470 

.1350 

.10 

.1280 

.0440 

.1320 

.05 

.1570 

.0280 

.1570 

.01 

.1930 

.0290 

.1930 

5 

lo 

1.00 

.0490 

.0500 

.0470 

.80 

.0460 

.0510 

.0570 

.60 

.0590 

.0750 

.1050 

.40 

.0810 

.0940 

.1300 

.20 

.0980 

.1020 

.1300 

.10 

.1700 

.1720 

.1950 

.05 

.1640 

.  1660 

.1730 

.01 

.1690 

.1690 

.1690 

5 

15 

1.00 

.0280 

.0450 

.0500 

.80 

.02$' 

.0-540 

.0880 

.60 

.0460 

.0990 

.1640 

.40 

.  07*  L 

.1140 

.2360 

.20 

.1200 

.1420 

.3530 

.10 

.1580 

.1660 

.4090 

.05 

.  1820 

.1820 

.4740 

.01 

.1920 

.1960 

.5080 

P:  N ( 0 , 

1) 

Q:  N(0, 

9) 

n 

m 

9 

K  -  S 

M  -  W 

T 

5 

5 

1.000 

.0440 

.0530 

.0550 

3.100 

.0400 

.0430 

.0490 

6.390 

.0730 

.0650 

.0760 

13.160 

.0850 

.0470 

.0870 

40.830 

.1130 

.0430 

.1150 

5 

10 

1.000 

.0530 

.0490 

.0470 

2.230 

.0420 

.0510 

.0590 

4.010 

.0610 

.0750 

.0870 

7.550 

.0750 

.0830 

.1090 

21.780 

.1100 

.1160 

.1510 

5 

15 

1.000 

.0320 

.0560 

.0630 

2.050 

.0460 

.0690 

.0790 

3.540 

.0380 

.0690 

.0990 

6.470 

.0760 

.1130 

.1630 

18.260 

.1030 

.1260 

.2550 

Table  5 


Estimated  Power  of  Two-Sample  One-Sided  Tests 
Constant  coefficient  of  variation  =  jj-  =  .5 

P:  N( .2, ( . 4 ) 2) 

Q:  N(p,  a2) 


K  -  S 

M  -  W 

T 

.20 

.0450 

.0530 

.0620 

.50 

.2110 

.1740 

.2470 

1.00 

.6770 

.5130 

.6970 

2.00 

.9920 

.9140 

.9930 

.20 

.0260 

.0470 

.0500 

.50 

.2170 

.2770 

.3440 

1.00 

.6940 

.7160 

.7840 

2.00 

.9890 

.9890 

.9920 

.20 

.0280 

.0510 

.0550 

.50 

.2480 

.3210 

.4380 

1.00 

.6960 

.7460 

.9030 

2.00 

.9940 

.9940 

.9990 

